Hdr image capture and display system for enhanced real-time welding visualization and assistance

ABSTRACT

A method, apparatus and system for enhanced welding visualization include splitting incoming light from the welding environment into at least a first optical path and a second optical path having different light levels using at least one beam splitter. Images of the split light having different light levels are captured using a respective imaging sensor. The images from the respective imaging sensors are fused to create a left-eye fused image and a right-eye fused image. The left-eye fused image is displayed on a display at a location of a left eye of a user and the right-eye fused image is displayed on a display at a location of a right eye of the user to provide a parallax-free, high dynamic range, representation of the welding environment.

FIELD

Embodiments of the present principles generally relate to weldingenvironments, and more particularly, to methods, apparatuses, andsystems for providing enhanced real-time welding visualization andassistance.

BACKGROUND

To date the majority of welding processes are performed manuallyhowever, welding-related industries, such as the manufacturing industryand the construction industry, are facing an apparent shortage ofexperienced/skilled welders. That is, a rapidly retiring workforce inconcurrence with the slow pace of traditional instructor-based weldertraining can be attributed as a cause of such shortage ofexperienced/skilled welders.

Fully automated/robotic approaches provide one way to address theshortage of experienced/skilled welders however, automated/roboticdevices can be relatively more expensive and the set up and tear down ofthe equipment can be cumbersome, reducing any efficiencies. In addition,there are areas in which automated/robotic devices cannot fit.

In some current day systems designed to provide welding assistance,filters are used to provide images of, for example a weldingenvironment, having different dynamic ranges. However, filters are notvery durable or accurate and can wear quickly requiring frequentreplacement. In addition, in those and other systems images captured bycameras are merely combined to produce an image/video of, for example, awelding environment having an expanded dynamic range. However, a simplecombination of camera images can result in a loss of details of each ofthe images. For example, U.S. Pat. No. 9,918,018, issued Mar. 13, 2018to Beeson, teaches dynamic range enhancement methods and systems fordisplay for use in welding applications that uses filters for providingimages of, for example a welding environment, having different dynamicranges and also only teaches combining images captured by differentcameras to produce an image/video having an expanded dynamic range. Theinvention of U.S. Pat. No. 9,918,018 suffers such deficiencies describedabove.

SUMMARY

Embodiments of methods, apparatuses and systems for enhanced real-timewelding visualization and assistance are disclosed herein.

In some embodiments in accordance with the present principles, anapparatus for enhanced real-time welding visualization in a weldingenvironment includes a beam splitter splitting incoming light from thewelding environment into at least a first optical path and a secondoptical path having different light levels, the at least first andsecond optical paths each comprising a respective imaging sensor, acontrol unit receiving and fusing the images from each of the respectiveimaging sensors in the at least first optical path and second opticalpath to create a left eye fused image and a right eye fused image, and adisplay assembly displaying the left eye fused image at a location of aleft eye of a user of the apparatus and the right eye fused image at alocation of a right eye of the user of the apparatus.

In some embodiments in accordance with the present principles, theapparatus can further include a left eye imaging assembly and a righteye imaging assembly, each of the left eye imaging assembly and theright eye imaging assembly including at least one beam splittersplitting incoming light from the welding environment into at least afirst optical path and a second optical path having different lightlevels, the at least first and second optical paths each comprising arespective imaging sensor, where images captured in the left eye imagingassembly are fused and displayed at the location of the left eye of theuser of the apparatus and images captured in the right eye imagingassembly are fused and displayed at the location of the right eye of theuser of the apparatus to provide a three dimensional representation ofthe welding environment.

In some embodiments in accordance with the present principles, a methodfor enhanced real-time welding visualization in a welding environment,includes, in a wearable welding apparatus, splitting incident light fromthe welding environment into at least a first optical path and a secondoptical path having different light levels using at least one beamsplitter, in each of the at least first optical path and second opticalpath, capturing images of the respective, split light using an imagingsensor, fusing the images from the respective imaging sensors of theleast first optical path and second optical path of the left eye imagingassembly and the right eye imaging assembly to create a left eye fusedimage and a right eye fused image, and displaying the left eye fusedimage on a display at a location of a left eye of a user of the wearablewelding apparatus and the right eye fused image on a display at alocation of a right eye of the user of the wearable welding apparatus.

In some embodiments in accordance with the present principles, themethod can further include splitting the incident light from the weldingenvironment into at least a first optical path and a second optical pathhaving different light levels using at least one beam splitter in eachof a left eye imaging assembly and a right eye imaging assembly andfusing and displaying images captured in the left eye imaging assemblyat the location of the left eye of the user of the apparatus and fusingand displaying images captured in the right eye imaging assembly at thelocation of the right eye of the user to provide a three dimensionalrepresentation of the welding environment.

In some embodiments in accordance with the present principles, a methodfor enhanced real-time welding visualization and assistance in a weldingenvironment includes splitting incoming light into a first optical pathhaving a minority of the split light and a second optical path having amajority of the split light using a beam splitter, and in the firstoptical path, focusing the minority of the split light from the beamsplitter using a high f-number lens, filtering the focused light fromthe high f-number lens using a neutral density filter, and capturingimages of the filtered light from the neutral density filter using afirst camera. In some embodiments the method can further include, in thesecond optical path, focusing the majority of the split light from thebeam splitter using a low f-number lens, capturing images of the lightfrom the low f-number lens using a second camera. The method can furtherinclude fusing the images from the first camera and the second camera,and displaying at least a portion of the fused images as left eyedisplay images and right eye display images.

Other and further embodiments in accordance with the present principlesare described below.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentprinciples can be understood in detail, a more particular description ofthe principles, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments in accordance with the present principles and aretherefore not to be considered limiting of its scope, for the principlesmay admit to other equally effective embodiments.

FIG. 1a depicts a high level block diagram of an advanced weldingmask/helmet system for enhanced real-time welding visualization andassistance in accordance with a first embodiment of the presentprinciples.

FIG. 1b depicts a high level block diagram of an advanced weldingmask/helmet system for enhanced real-time welding visualization andassistance having three-dimensional imaging capabilities in accordancewith a second embodiment of the present principles.

FIG. 2 depicts a high level block diagram of the inside of themask/helmet of the advanced welding mask/helmet system of FIG. 1 inaccordance with an embodiment of the present principles.

FIG. 3 depicts a high level block diagram of an imaging assemblysuitable for use in the welding mask/helmet system of FIG. 1 inaccordance with an embodiment of the present principles.

FIG. 4 depicts a pictorial depiction of an image of a weldingenvironment captured by an advanced welding mask/helmet system inaccordance with an embodiment of the present principles.

FIG. 5a depicts a source image of a welding environment includingwelding sparks, captured, for example, by a first imaging sensor in afirst optical path of at least one of a left-eye imaging assembly and aright-eye imaging assembly of a welding mask/helmet system in accordancewith an embodiment of the present principles.

FIG. 5b depicts a binary saturation mask generated from the source imageof FIG. 5a in accordance with an embodiment of the present principles

FIG. 6 depicts a high level block diagram of a belt assembly for housingthe control unit and a battery for powering the control unit inaccordance with an embodiment of the present principles.

FIG. 7 depicts a high level block diagram of a control unit suitable foruse in the welding mask/helmet systems of FIG. 1a and FIG. 1b inaccordance with an embodiment of the present principles.

FIG. 8 depicts a flow diagram of a method for enhanced real-time weldingvisualization in accordance with an embodiment of the presentprinciples.

FIG. 9 depicts a flow diagram of a method 900 for enhanced real-timewelding visualization in accordance with a stereo/three-dimensionalembodiment of the present principles.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures. The figures are not drawn to scale and may be simplifiedfor clarity. It is contemplated that elements and features of oneembodiment may be beneficially incorporated in other embodiments withoutfurther recitation.

DETAILED DESCRIPTION

Embodiments of the present principles generally relate to methods,apparatuses and systems for providing enhanced real-time weldingvisualization and assistance. While the concepts of the presentprinciples are susceptible to various modifications and alternativeforms, specific embodiments thereof are shown by way of example in thedrawings and are described in detail below. It should be understood thatthere is no intent to limit the concepts of the present principles tothe particular forms disclosed. On the contrary, the intent is to coverall modifications, equivalents, and alternatives consistent with thepresent principles and the appended claims. For example, althoughembodiments of the present principles will be described primarily withrespect to a particular welding mask/helmet system having specificcomponents in a particular welding environment, such teachings shouldnot be considered limiting. Embodiments in accordance with the presentprinciples can be implemented within substantially any weldingmask/helmet in substantially any welding environment within the conceptsof the present principles.

Embodiments in accordance with the present principles provide a weldingmask/helmet system including a welding mask/helmet integrated with oneor multiple sensors including but not limited to cameras, acousticmicrophones, infrared sensors, thermal cameras, accelerometers, and GPSdevices. In addition, the welding mask/helmet can include communicationmeans and accept data from one or multiple external sensors. In variousembodiments, sensor information can be displayed on a screen that can beintegrated within the welding mask/helmet and, in some embodiments, canbe integrated within a visor portion of the welding mask/helmet.

In various embodiments in accordance with the present principles, in awelding helmet/mask system, incident light from a welding environment issplit into at least two optical paths (i.e., a plurality of opticalpaths) of varying light levels. In each of the at least two opticalpaths, the light of the varying light levels is captured by a respectiveimaging sensor to create images/video streams of the welding environmenthaving different exposure levels. That is, in some embodiments, imageframes of a welding environment are captured using different opticalpaths having different light levels. The dark and bright scene detailsof the welding environment can be captured by differentiating the lightlevels in each of the optical paths by using beam splitters, usingdifferent f-number lenses or placing different neural density filters insome or all of the optical paths. In some embodiments requiringstereo-vision, in each of a left-eye imaging assembly and a right-eyeimaging assembly, incident light from a welding environment is splitinto at least two optical paths (i.e., a plurality of optical paths) ofvarying light levels. In each of the at least two optical paths in eachof a left-eye imaging assembly and a right-eye imaging assembly, thelight of the varying light levels is captured by a respective imagingsensor to create images/video streams of the welding environment havingdifferent exposure levels.

Advantageously because of the architecture of a welding helmet/masksystem in accordance with the present principles, image frames of thewelding environment can be captured at the same time, but the intensityof each frame is differentiated by at least one of a beam splitter,f-number lenses or by neutral density filters. Such components areimplemented in accordance with the present principles to control anamount of light energy captured by respective imaging sensors.

In addition or alternatively, in some embodiments, at least one imagingsensor can be configured to capture images of the welding environment atdifferent frame rates to capture images with different light levels. Forexample, in some embodiments, at least one imaging sensor can beconfigured to capture images of the welding environment at a higherframe rate. In some embodiments, the imaging sensor can run double orhigher of the display rate, and each of the image frames can beconfigured with different exposure times, such that dark scenes whichrequire high exposure time, and bright scenes which require low exposuretime, can be captured in sequence. Although in such embodiments, imageframes are not captured at the same time, because they are captured atleast double of the display rates, the spatial difference can be ignoredfor welding systems.

The respective images captured by the imaging sensors of the at leasttwo optical paths or the at least four optical paths for an embodimentincluding stereo/three-dimensional capabilities including each of aleft-eye imaging assembly and a right-eye imaging assembly, are fused bya processor, of for example a control unit, into fused left-eye andright-eye images of the welding environment having the details of eachof the different exposure images/video streams of the weldingenvironment. The fused left-eye images are displayed on a left-eyedisplay to be presented to a left-eye of a user of the helmet/masksystem of the present principles and the fused right-eye images aredisplayed on a right-eye display to be presented to a right-eye of theuser of the helmet/mask system of the present principles.

In some embodiments, the light from the welding environment is splitinto the at least two optical paths having different light levels by atleast one respective beam splitter. In some other embodiments, the lightfrom the welding environment is split into a plurality of optical pathshaving different light levels by more than one respective beam splitter.The split light in each of the optical paths can be directedperpendicularly to the respective imaging sensor in each of the opticalpaths. Advantageously, the architecture of a welding mask helmet/masksystem in accordance with the present principles and specifically thearrangement of the beam splitter, the multiple optical paths havingvarying light levels, and the respective imaging sensors, provides aparallax-free, high dynamic range visualization and video acquisitionsystem. In addition, in some embodiments, the architecture of a weldingmask helmet/mask system in accordance with the present principles andspecifically the respective imaging sensors of the left-eye imagingassembly and the right-eye imaging assembly, advantageously provideseparate left-eye images and right-eye images, providing athree-dimensional representation of the welding environment to optimizedepth perception.

Some embodiments in accordance with the present principles furtherprovide welding assistance to a user of an advanced welding mask/helmetsystem of the present principles. That is, in some embodiments,information and images can be presented to a user of an advanced weldingmask/helmet system of the present principles in the form of at leastaudio and images presented on a display to assist a welder in theperformance of a weld.

It should be noted that the term light intensity as used throughout thisdisclosure is intended to encompass both the principles of lightintensity and dynamic range.

FIG. 1a depicts a high level block diagram of an advanced weldingmask/helmet system 100 in accordance with an embodiment of the presentprinciples. The advanced welding mask/helmet system 100 of FIG. 1illustratively comprises a welding mask/helmet 120 and a control unit130. The welding mask/helmet 120 of FIG. 1 illustratively comprises animaging assembly 122 and an optional sensor 126. FIG. 1 further depictsa remote welding station 140, which contains welding equipment andaccessories for performing welding operations. Although in FIG. 1 a, theadvanced welding mask/helmet system 100 is depicted as comprising asingle sensor 126 (e.g., the optional sensor) and the optional sensor126 is depicted as being mounted on the welding mask/helmet 120, inother embodiments in accordance with the present principles, a weldingmask/helmet system can comprise more than one sensor (described ingreater detail below), with some sensors able to be mounted on thewelding mask/helmet 120 and other sensors comprising separatecomponents.

FIG. 1b depicts a high level block diagram of an advanced weldingmask/helmet system 100 in accordance with a second,stereo/three-dimensional embodiment of the present principles. Theadvanced welding mask/helmet system 160 of FIG. 1b illustrativelycomprises a welding mask/helmet 120 and a control unit 130. The weldingmask/helmet 120 of FIG. 1b illustratively comprises a left-eye imagingassembly 182, a right-eye imaging assembly 184, and an optional sensor186. FIG. 1b further depicts a remote welding station 140, whichcontains welding equipment and accessories for performing weldingoperations. Although in FIG. 1b , the advanced welding mask/helmetsystem 160 is depicted as comprising a single sensor 186 (e.g., theoptional sensor) and the optional sensor 186 is depicted as beingmounted on the welding mask/helmet 120, in other embodiments inaccordance with the present principles, a welding mask/helmet system cancomprise more than one sensor (described in greater detail below), withsome sensors able to be mounted on the welding mask/helmet 120 and othersensors comprising separate components.

FIG. 2 depicts a high level block diagram of the inside of themask/helmet 120 of the advanced welding mask/helmet system 100/160 ofFIGS. 1a and 1b in accordance with an embodiment of the presentprinciples. As depicted in FIG. 2, the mask/helmet 120 can comprise adisplay 150 on the inside for presenting images, video streams and dataand information to a user of the mask/helmet 120. In the embodiment ofFIG. 2, the display 150 of the mask/helmet 120 comprises a head mounteddisplay (HMD) 150 mounted inside of the mask/helmet 120. In someembodiments, the HMD 150 comprises adjustable display positioning tomeet a user's needs concerning inclination as well as a distance betweenthe left and right displays of the HMD 150. As depicted in FIG. 2, thedisplay 150 of the mask/helmet 120 can include a first viewing means(e.g., display) 151 at a location of a left eye of a user and a secondviewing means (e.g., display) 152 at a location of a right eye of auser.

FIG. 3 depicts a high level block diagram of an imaging assembly 300suitable for use in the welding mask/helmet system 100 of FIG. 1 a andthe welding mask/helmet system 160 of FIG. 1 b as the imaging assembly122 or, in a stereo/three-dimensional embodiment, as the left-eyeimaging assembly 172 and the right-eye imaging assembly 174, inaccordance with an embodiment of the present principles. The imagingassembly 300 of FIG. 3 illustratively comprises a first camera lens 305,a second camera lens 310, a beam splitter 315, an optional filter 320, afirst camera 350 and a second camera 360 (illustratively printed circuitboard cameras).

In the embodiment of the imaging assembly 300 FIG. 3, incident lightfrom the welding environment is split by the beam splitter 315 into,illustratively, two optical paths. The beam splitter 315 of theembodiment of FIG. 3 illustratively comprises a beam splitter thatsplits incident light into a first optical path 311 comprising amajority of the light and a second optical path 312 comprising aminority of the light. For example, in the imaging assembly 300 of theembodiment of FIG. 3, the beam splitter 315 can a 60% reflection and 40%transmission beam splitter. That is, in the embodiment of the imagingassembly 300 FIG. 3, 40% of incident light travels through the beamsplitter 315 along the first optical path 311 to the first camera lens305 and 60% of incident light is reflected to the right along the secondoptical path 312 to the second camera lens 310. As such, in the imagingassembly 300 of FIG. 3, each of the two optical paths 311, 312 comprisedifferent light levels.

In the embodiment of the imaging assembly 300 of FIG. 3, the firstcamera lens 305 illustratively comprises a high f-number lens andillustratively an f/6 lens and the second camera lens 310 illustrativelycomprises a low f-number camera lens and illustratively an f/2 lens. Inthe embodiment of FIG. 3, the first, f/6 camera lens 305 receives andfocuses the 40% transmitted light from the beam splitter 315 and thesecond, f/2 camera lens 310 receives and focuses the 60% reflected lightfrom the beam splitter 315. In the embodiment of FIG. 3, the differentf-number camera lenses 305, 310 further differentiate the light levelsin the two optical paths 311, 312.

In the embodiment of the imaging assembly 300 of FIG. 3, in the firstoptical path 311, the 40% of the light focused by the first, highf-number camera lens 305 can be filtered by the optional filter 320(illustratively a neutral density filter). In the embodiment of FIG. 3,the optional filter 320 further differentiates the light levels in thetwo optical paths 311, 312.

In the embodiment of the imaging assembly 300 of FIG. 3, the 40% of thefocused light filtered by the optional filter 320 is captured and imagedby the first camera 350 and the 60% of the light focused by the secondlens 310 is captured and imaged by the second camera 360 creatingimages/video streams of the welding environment having two differentlight levels.

For example, in the imaging assembly 300 of FIG. 3, the first camera 350collects much less light than the second camera 360 because the beamsplitter 315 only transmits 40% of incident light to the first camera350 and because the optional filter 320 (illustratively a 1.2 ND filter)allows only approximately 9% of light to pass to the first camera 350.In addition, in the imaging assembly 300 of FIG. 3 the first camera lens305 collecting light to be directed to the first camera 350 isillustratively an f/6 lens, which accepts approximately eight (8) timesless light than the second camera lens 310, which in the imagingassembly 300 of FIG. 3 is illustratively an f/2 lens.

Although the embodiment of the imaging assembly 300 of FIG. 3illustratively comprises a beam splitter splitting incident light from awelding environment into two optical paths, a first optical path 311comprising a majority of the split light and a low f-number lens and asecond optical path 312 comprising a minority of the split light, a highf-number lens and a filter, the illustrated embodiment should not beconsidered limiting. The imaging assembly 300 of FIG. 3 is intended todepict an embodiment of at least one imaging assembly comprising one ormore beam splitters splitting incident light into at least two opticalpaths having varying light levels for providing a parallax-free, highdynamic range video acquisition system creating images/video streams ofthe welding environment having two respective exposure levels enablingboth, bright and dark scene details to be captured, in some embodiments,at the same time. In other embodiments of the present principles,incident light from a welding environment can be split by one or morebeam splitters into a plurality of optical paths having different lightlevels ultimately creating images/video streams of the weldingenvironment having a plurality of respective exposure levels inaccordance with the present principles enabling both, bright dark andin-between light level scene details to be captured, in someembodiments, at the same time.

In addition or alternatively, as described above, in some embodiments inaccordance with the present principles, at least one imaging sensor(e.g., camera) can be configured to capture images of the weldingenvironment at a higher frame rate. In such embodiments, the imagingsensor can run double or higher of the display rate, and each of theimage frames can be configured with different exposure times, such thatdark scenes which require high exposure time, and bright scenes whichrequire low exposure time, can be captured. In such embodiments, imagesof the welding environment can be captured at multiple frame rates withvarious exposure times such that scenes of the welding environmenthaving light levels between very dark scenes and very bright scenes canalso be captured.

In accordance with the above described embodiments, in each imagingassembly of the present principles, such as the imaging assembly 300FIG. 3, the video streams captured by the two imaging sensors (e.g.,camera 350 and camera 360) are communicated to the control unit 130. Atthe control unit 130, the video streams are combined into a left-eyeimage(s)/video stream and a right-eye image(s)/video stream using fusiontechnology. In some embodiments in accordance with the presentprinciples, Laplacian Pyramid Fusion technology from SRI Internationalin Menlo Park, Calif. is implemented for fusing the four video streams,the Laplacian Pyramid Fusion technology being the subject of U.S. Pat.No. 8,411,938 entitled MULTI-SCALE MULTI-CAMERA ADAPTIVE FUSION WITHCONTRAST NORMALIZATION and assigned to SRI International, which isincorporated herein by reference in its entirety.

In some embodiments, the fused video streams result in an 8-bit outputstream capable of being displayed on a display such as a computermonitor or a head-mounted display while retaining all of the detailsoriginally in the four separate video streams. That is, extremely brightdetails of, for example, a welding arc and objects near the welding arc,extremely dark details, for example, the background of the weldingenvironment, and images having a brightness in between are preserved inthe fused output.

For example, FIG. 4 depicts a pictorial depiction of an image of awelding environment captured by an advanced welding mask/helmet systemin accordance with an embodiment of the present principles. In theembodiment of FIG. 4, it is clear that in an image captured by anadvanced welding mask/helmet system in accordance with an embodiment ofthe present principles, the welding torch 402, the welding wire 404, thewelding arc 406, the welding puddle 408, welding sparks 410, a glowingbead of a weld that just solidified 412, the bead of the weld as thebead cools down and no longer glows 414, a welding sample 416, and thebackground 418 of the welding environment are all simultaneously visibleon a display, such as the HMD 150, and visible by a user of an advancedwelding mask/helmet system of the present principles.

In some embodiments, an image fusion process first decomposes eachsource image into multi-spatial, multi-band images, which are called apyramid of images. Each image in a pyramid corresponds to a narrowfrequency band. Image fusion enforces a selection rule that selects bestfeatures per pixel and per band among all source images. The localselection is based on the strength of salient features, the confidenceof the pixels from the source images. For example, if an area in asource image is saturated, or is full of noise, the pixels in that areahas zero confidence, then in the selection of features in thecorresponding locations per band, pixels pertaining to saturation andnoise are not selected. The selection process outputs one image perband, which maintains the best features per frequency band. These bandedimages thus form a fused pyramid. Lastly, a reverse process is performedas referenced to decomposition is to reconstruct the fused pyramid intoa single fused image. This fused image contains the best valid localstructures from each of the source images.

Referring back to FIGS. 1b -3, in some stereo/three-dimensionalembodiments in accordance with the present principles, the output streamof the left eye imaging assembly 122 is displayed on the HMD 150 of thewelding mask/helmet 120 of the advanced welding mask/helmet system 100of FIG. 1 such that the output stream of the left eye imaging assembly122 is visible by a left eye of a user of the welding mask/helmet 120.Similarly, in some embodiments in accordance with the presentprinciples, the output stream of the right eye imaging assembly 124 isdisplayed on the HMD 150 of the welding mask/helmet 120 of the advancedwelding mask/helmet system 100 of FIG. 1 such that the output stream ofthe right eye imaging assembly 124 is visible by a right eye of a userof the welding mask/helmet 120. In some alternate embodiments inaccordance with the present principles, a display of the weldingmask/helmet 120 comprises a single display upon which streams aredisplayed at appropriate locations to be viewed by an appropriate eye ofa user. In some other embodiments in accordance with the presentprinciples, a display of the welding mask/helmet 120 comprises adedicated display for each eye of a user.

The dedicated capture and assignment of images/video streams for displayto a specific eye of a user of the welding mask/helmet 120 in accordancewith the present principles, results in a true stereoscopic (e.g., 3D)visualization for a user. To preserve a realistic depth perception, insome embodiments in accordance with the present principles, the firstimaging assembly 122 and the second imaging assembly 124 are physicallymounted and oriented on the welding mask/helmet 120 of the weldingmask/helmet system 160 of FIG. 1b so as to approximate the pose andintra-ocular separation of a human welder.

In some embodiments of the welding mask/helmet system in accordance withthe present principles, during welding and specifically during a periodin which the welding arc is on, in each imaging assembly, such as theimaging assembly 300 of FIG. 3, images of the welding environment areacquired by the first camera 350 through the beam splitter 315, the highf-number first camera lens 305 and the filter 320. More generally,during periods of extreme brightness, images/video streams of a weldingenvironment are captured using components in optical paths, whichultimately comprise much less light intensity than components andoptical path comprising more light intensity. Conversely, during periodsof relatively greater darkness and specifically during a period in whichthe welding arc is off, images of the welding environment are acquiredby the second camera 360 through the beam splitter 315. More generally,during periods of darkness, images/video streams are captured usingcomponents in optical paths which ultimately comprise much more lightintensity than components and an optical path comprising less lightintensity.

The architecture of the welding mask/helmet system 100 of FIG. la and160 of FIG. 1b in accordance with the present principles, andspecifically the arrangement of the beam splitter 315 and the first andsecond camera 350, 360 of the imaging assembly 300, provides aparallax-free, high dynamic range video acquisition system for a weldingenvironment.

In addition, advantageously, because of the architecture of the weldingmask/helmet system 100 of FIGS. 1 a and 160 of FIG. 1 b in accordancewith embodiments of the present principles, a user is able to keepwearing the welding mask/helmet 120 of the welding mask/helmet system100/160 during both, periods of extreme brightness and extreme darknessin the welding environment and still be able to see a clear image of thewelding environment. More specifically and referring back to FIGS. 1-3,in some embodiments in accordance with the present principles, theoptional sensor 126/186 of the welding mask/helmet system 100 of FIG. 1aand the welding mask/helmet system 160 of FIG. 1b can include a lightsensor 126/186 which can be mounted on the welding mask/helmet 120. Insome embodiments, the light sensor 126/186 senses periods of brightnessin the welding environment (i.e., when a weld arc is on) andcommunicates a signal to the control unit 130 indicating the existenceof the brightness in the welding environment. In some other embodiments,the optional light sensor 126/186 senses periods of darkness in thewelding environment (i.e., when a weld arc is off) and communicates asignal to the control unit 130 indicating the existence of the darknessin the welding environment.

Upon receiving a signal from the light sensor 126/186, the control unit130 determines, based on the signal received, from which imaging sensor(e.g., the first camera 350 or the second camera 360) to displaycaptured image(s)/video stream on a display of the welding mask/helmet120. For example and as described above, in some embodiments, duringperiods of extreme brightness in the welding environment, in response toa signal received from the optional light sensor, the control unit 130causes a video stream captured by the first camera 350 to be displayedon a display of the welding mask/helmet 120 for a respective eye of auser. Conversely and as described above, in some embodiments, duringperiods of extreme darkness in the welding environment, in response to asignal received from the optional light sensor, the control unit 130causes a video stream captured by the second camera 360 to be displayedon a display of the welding mask/helmet 120 for a respective eye of auser.

Alternatively or in addition, in some embodiments in accordance with thepresent principles, a signal captured by at least one of the firstcamera 350 and the second camera 360 can be implemented by, for example,the control unit 130 to determine from which camera (e.g., the firstcamera 350 or the second camera 360) to display a captured video streamon a display of the welding mask/helmet 120. For example, if a capturedvideo stream from any of the four cameras (e.g., the first camera 350and the second camera 360) indicates a presence of brightness in thewelding environment, the control unit 130, having access to the capturedvideo streams of each of the four cameras, causes a video streamcaptured by the first camera 350 (which collects less light) to bedisplayed on a display of the welding mask/helmet 120 for a respectiveeye of a user. Conversely, if captured video streams of the four cameras(e.g., the first camera 350 and the second camera 360) indicate apresence of darkness in the welding environment, the control unit 130causes a video stream captured by the second camera 360 to be displayedon a display of the welding mask/helmet 120 for a respective eye of auser.

In some embodiments in accordance with the present principles, sparksgenerated in a welding environment by, for example, arc welding can beeliminated from a display of a video stream. More specifically, sparksevident in an image or images captured by one of the cameras (e.g., thefirst camera 350 and the second camera 360) can be identified by thecontrol unit 130, for example, when processing the captured videostreams. At the control unit 130, the portions (e.g., pixels) of theimages containing sparks can be eliminated from the images. For example,in some embodiments in accordance with the present principles, signalsof the camera pixels containing sparks can be eliminated and an averageof signals from surrounding/neighboring pixels can replace theeliminated signals.

For example, FIGS. 5a and 5b depict a pictorial representation of aspark removal process in accordance with an embodiment of the presentprinciples. FIG. 5a depicts a first source image of a weldingenvironment including welding sparks, captured, for example, by a firstimaging sensor in a first optical path of a welding mask/helmet systemin accordance with the present principles. In the embodiment of FIGS. 5aand 5b , a binary saturation mask, as depicted in FIG. 5b , is generatedfrom the source image of FIG. 5a . The individual sparks are thendetected from the mask image and the area of each spark is computed.Based on a predetermined threshold size value, saturated regions in thesource image corresponding to the welding tip and the background of thewelding environment are excluded from the binary saturation mask. Valuessurrounding the sparks are then determined. For example, in someembodiments, the contour for each spark is enlarged with morphologicaloperations in the binary mask image. Values along the contours of aspark in the image are accumulated and an average contour value perspark is calculated. A pixel value of each spark is then replaced withthe calculated average value.

In some embodiments in accordance with the present principles, at leastone of the imaging sensors (e.g., the first camera 350 and the secondcamera 360) comprises at least one near infrared (NIR) camera. In suchembodiments, a welding mask/helmet system in accordance with the presentprinciples is able to provide a clear image of a welding environmenteven if the welding environment is filled with smoke.

Referring back to FIG. la and 1 b, in some embodiments in accordancewith the present principles, the optional sensor 126/186 of the weldingmask/helmet system 100/160 can comprise at least one temperature sensorsuch as a low resolution infrared camera or thermal sensor. That is, aparameter that is particularly important to detect in welding is notonly the temperature in the actual melting zone but also the heatdistribution in the surrounding material. In embodiments of a weldingmask/helmet system 100/160 which include at least one infrared camera126/186 or thermal sensor for sensing temperature, the at least oneinfrared camera or thermal sensor 126/186 can sense temperature of theactual melting zone in the welding environment and also the heatdistribution in the surrounding material. The data collected by the atleast one infrared camera or thermal sensor 126/186 is communicated tothe control unit 130. The control unit 130 can cause the temperaturedata to be displayed on the HMD 150 of the welding mask/helmet 120 in anarbitrary manner. For example, in some embodiments, the temperature datacan be displayed as numerical data, diagrams, by coloring differenttemperature ranges differently which are shown in an image representinga weld and the surrounding zone, via a heat map or a combination of anyof the above.

In some embodiments, the temperature information can be displayed on adedicated part of a display either over an image or video stream of thewelding environment or outside of a display of an image or video streamof a welding environment.

In some embodiment in accordance with the present principles, theoptional sensor 126/186 can include at least one of a camera, acousticmicrophone, infrared sensor, thermal camera, accelerometer, and a GPSdevice for providing respective information to the control unit 130 tobe used for providing information and welding assistance to a user of awelding mask/helmet system of the present principles at least asdescribed herein with respect to any sensor.

In various embodiments in accordance with the present principles, thecontrol unit 130 of the welding mask/helmet system 100/160 of FIGS. 1aand 1b can enable a user to interact with the welding mask/helmet system100 and with the welding environment for example, via the weldingstation 140. For example, in some simple embodiments, the control unit130 can cause a presentation of information related to the weldingenvironment and the welding station 140 to be presented to a user viathe HMD 150. For example, in some embodiments, using integrated orexternal thermal sensors, a temperature of the objects that are beingwelded can be monitored and the control unit 130 can cause the displayof such information on the HMD 150 as, for example, a thermal image isoverlaid on the video stream image of the welding environment such thata welder is able to make decisions about how to change a weld procedureor parameters so that a desired weld can be achieved. For example, awelder can adjust a dwell time at a particular site in response to thetemperature of the objects being welded. Other welding parameters thatcan be displayed to and adjusted by a welder can include but are notlimited to a feed speed of a weld wire, a position of a weld wire, aposition of the weld torch, a current or voltage of a power supplyassociated with the weld torch and the like.

In some embodiments in accordance with the present principles, thecontrol unit 130 can communicate with the welding station 140 todetermine parameters of the welding environment for a least displayingsuch parameters to a user of the welding mask/helmet system 100/160 via,for example, the HMD 150. In some embodiments, the control unit 130 cancommunicate with the welding station 140 via a wired connection, such asUSB or HDMI or alternatively or in addition, the control unit 130 cancommunicate with the welding station 140 via a wireless connection, suchas Bluetooth or Wi-Fi. Having such information, such as weldingparameters, displayed on for example the HMD 150, a welder is able tomake decisions about how to change a weld procedure or parameters sothat a desired weld can be achieved.

In some embodiments in accordance with the present principles,image(s)/video stream(s) of the weld process captured by at least one ofthe cameras (e.g., the first camera 350 and the second camera 360) canbe recorded by, for example, the control unit 130 in a memory of eitherthe control unit 130or an external memory. The quality of the weld canbe determined by analyzing the recorded images of the weld at thecontrol unit 130. For example, in some embodiments, recorded images ofthe weld process and/or the completed weld can be compared at thecontrol unit 130to stored images of what a proper weld process orcompleted weld should look like. The data regarding the weld quality canbe used for various purposes including but not limited to machinelearning, providing feedback regarding weld quality, and reporting orfor redoing the welds if such a need arises. Feedback provided regardingweld quality can be provided to a user on a display of a weldingmask/helmet system in accordance with embodiments of the presentprinciples. A user can use such feedback to change a weld procedure orparameters so that a desired weld can be achieved.

In some embodiments of a welding mask/helmet system, such as the weldingmask/helmet system 100 of FIG. 1 a and the welding mask/helmet system160 of FIG. 1 b, augmented reality can be used to assist a welder/userin performing an improved weld. For example, before the commencement ofa welding procedure, the control unit 130 can cause a display of a pathalong which the weld should occur on, for example, the HMD 150 forviewing by a welder. The path for a weld can be computed by, for examplethe control unit 130 or other control unit, based on knowledge of thegeometry of the weld, the material properties of the materials involved,stored knowledge of the path of such welds and knowledge gained fromprevious similar welds.

During the welding, the determined path can be displayed to awelder/user over an area of the image/video stream of the weldenvironment at which the weld is to be made. The video stream of thewelding by the welder can be monitored by the control unit 130 todetermine if the determined path is being followed by the welder duringthe welding process. If any deviation from the previously calculatedpath is determined, a warning can be provided to the welder in the formof a visual cue on, for example, the HMD 150.

In some embodiments in accordance with the present principles, thewelding mask/helmet system 100 of FIG. 1 a and the welding mask/helmetsystem 160 of FIG. 1b can further comprise an assembly for housing atleast the control unit 130 and a power source for supplying power to thecontrol unit 130. For example, FIG. 6 depicts a high level block diagramof a belt assembly 600 for housing the control unit 130 and a battery610 for powering the control unit in accordance with an embodiment ofthe present principles. The belt assembly 600 of FIG. 6 comprises acompartment 620 for holding the battery 610 and fastening holes(illustratively four fastening holes, collectively 630) for attachingthe control unit 130 to the belt assembly 600 via fasteners (not shown).The belt assembly 600 of FIG. 6 further comprises belt slots((illustratively four belt slots, collectively 640) for securing a belt(not shown) to the belt assembly 600 such that a welder is able tosecure the belt assembly 600 to, for example, the welder's waist using abelt (not shown).

FIG. 7 depicts a high level block diagram of a control unit 130 suitablefor use in the welding mask/helmet system 100 of FIG. la and the weldingmask/helmet system 160 of FIG. 1b in accordance with an embodiment ofthe present principles. In some embodiments control unit 130 can beconfigured to implement the methods of the present principles asprocessor-executable executable program instructions 722 (e.g., programinstructions executable by processor(s) 710) in various embodiments.

In the embodiment of FIG. 7, control unit 130 includes one or moreprocessors 710 a-710 n coupled to a system memory 720 via aninput/output (I/O) interface 730. The control unit 130 further includesa network interface 740 coupled to I/O interface 730, and one or moreinput/output devices 750, such as cursor control device 760, keyboard770, and display(s) 780. In various embodiments, any of the componentscan be utilized by the system to receive user input described above. Invarious embodiments, a user interface can be generated and displayed ondisplay 780. In some cases, it is contemplated that embodiments can beimplemented using a single instance of the control unit 130, while inother embodiments multiple such systems, or multiple nodes making up thecontrol unit 130, can be configured to host different portions orinstances of various embodiments. For example, in one embodiment someelements can be implemented via one or more nodes of the control unit130 that are distinct from those nodes implementing other elements. Inanother example, multiple nodes can implement the control unit 130 in adistributed manner.

In different embodiments, the control unit 130 can be any of varioustypes of devices, including, but not limited to, a personal computersystem, desktop computer, laptop, notebook, tablet or netbook computer,mainframe computer system, handheld computer, workstation, networkcomputer, a camera, a set top box, a mobile device, a consumer device,video game console, handheld video game device, application server,storage device, a peripheral device such as a switch, modem, router, orin general any type of computing or electronic device.

In various embodiments, the control unit 130 can be a uniprocessorsystem including one processor 710, or a multiprocessor system includingseveral processors 710 (e.g., two, four, eight, or another suitablenumber). Processors 710 can be any suitable processor capable ofexecuting instructions. For example, in various embodiments processors710 may be general-purpose or embedded processors implementing any of avariety of instruction set architectures (ISAs). In multiprocessorsystems, each of processors 710 may commonly, but not necessarily,implement the same ISA.

System memory 720 may be configured to store program instructions 722and/or data 732 accessible by processor 710. In various embodiments,system memory 720 may be implemented using any suitable memorytechnology, such as static random-access memory (SRAM), synchronousdynamic RAM (SDRAM), nonvolatile/Flash-type memory, or any other type ofmemory. In the illustrated embodiment, program instructions and dataimplementing any of the elements of the embodiments described above canbe stored within system memory 720. In other embodiments, programinstructions and/or data can be received, sent or stored upon differenttypes of computer-accessible media or on similar media separate fromsystem memory 720 or the control unit 130.

In one embodiment, I/O interface 730 can be configured to coordinate I/Otraffic between processor 710, system memory 720, and any peripheraldevices in the device, including network interface 740 or otherperipheral interfaces, such as input/output devices 750. In someembodiments, I/O interface 730 can perform any necessary protocol,timing or other data transformations to convert data signals from onecomponent (e.g., system memory 720) into a format suitable for use byanother component (e.g., processor 710). In some embodiments, I/Ointerface 730 can include support for devices attached through varioustypes of peripheral buses, such as a variant of the Peripheral ComponentInterconnect (PCI) bus standard or the Universal Serial Bus (USB)standard, for example. In some embodiments, the function of I/Ointerface 730 can be split into two or more separate components, such asa north bridge and a south bridge, for example. Also, in someembodiments some or all of the functionality of I/O interface 730, suchas an interface to system memory 720, can be incorporated directly intoprocessor 710.

Network interface 740 can be configured to allow data to be exchangedbetween the control unit 130 and other devices attached to a network(e.g., network 790), such as one or more external systems or betweennodes of the control unit 130. In various embodiments, network 790 caninclude one or more networks including but not limited to Local AreaNetworks (LANs) (e.g., an Ethernet or corporate network), Wide AreaNetworks (WANs) (e.g., the Internet), wireless data networks, some otherelectronic data network, or some combination thereof. In variousembodiments, network interface 740 can support communication via wiredor wireless general data networks, such as any suitable type of Ethernetnetwork, for example; via digital fiber communications networks; viastorage area networks such as Fiber Channel SANs, or via any othersuitable type of network and/or protocol.

Input/output devices 750 can, in some embodiments, include one or moredisplay terminals, keyboards, keypads, touchpads, scanning devices,voice or optical recognition devices, or any other devices suitable forentering or accessing data by one or more computer systems. Multipleinput/output devices 750 can be present in the control unit 130 or canbe distributed on various nodes of the control unit 130. In someembodiments, similar input/output devices can be separate from thecontrol unit 130 and can interact with one or more nodes of the controlunit 130 through a wired or wireless connection, such as over networkinterface 740.

In some embodiments, the illustrated control unit 130 can implement anyof the operations and methods described above, such as the methodsillustrated by the flowcharts of FIG. 8 and FIG. 9 (described below). Inother embodiments, different elements and data can be included.

Those skilled in the art will appreciate that the control unit 130 ismerely illustrative and is not intended to limit the scope ofembodiments. In particular, the computer system and devices can includeany combination of hardware or software that can perform the indicatedfunctions of various embodiments, including computers, network devices,Internet appliances, PDAs, wireless phones, pagers, and the like. Thecontrol unit 130 can also be connected to other devices that are notillustrated, or instead can operate as a stand-alone system. Inaddition, the functionality provided by the illustrated components canin some embodiments be combined in fewer components or distributed inadditional components. Similarly, in some embodiments, the functionalityof some of the illustrated components may not be provided and/or otheradditional functionality can be available.

In some embodiments in accordance with the present principles, a userinterface to enable a user to interact with at least the control unit130 and to control parameters of the welding environment can be providedby the control unit 130. In some embodiments, the user interface can beimplemented as a menu driven application presented on a display of anadvanced welding mask/helmet system of the present principles, such asthe advanced welding mask/helmet system 100 of FIG. 1a and the advancedwelding mask/helmet system 160 of FIG. 1 b, and the one or moreinput/output devices of at least the control unit 130 can be used toprovide interaction between a user of an advanced welding mask/helmetsystem of the present principles and the user interface. In someembodiments, buttons or other control devices of a an assembly forhousing at least the control unit 130 and a power source for supplyingpower to the control unit 130, for example, the belt assembly 600 ofFIG. 6, can be implemented to provide interaction between a user of anadvanced welding mask/helmet system of the present principles and theuser interface.

FIG. 8 depicts a flow diagram of a method 800 for enhanced real-timewelding visualization in accordance with an embodiment of the presentprinciples. The method 800 begins at 802 during which, in a wearablewelding apparatus, such as the advanced welding mask/helmet system inaccordance with embodiments of the present principles, incident light ina welding environment is split using a beam splitter into at least afirst optical path and a second optical path having different lightlevels. The method can proceed to 804.

At 804, images of the respective, split light in each of the first andsecond optical paths are captured using a respective imaging sensor. Themethod 800 can proceed to 806.

At 806, images from the respective imaging sensors of the least firstoptical path and second optical path are fused to create a left eyefused image and a right eye fused image. The method 800 can proceed to808.

At 808, the left eye fused image is displayed at a location of a lefteye of a user of the wearable welding apparatus and the right eye fusedimage is displayed at a location of a right eye of the user of thewearable welding apparatus to provide a, high dynamic rangerepresentation of the welding environment and optimize depth perception.The method 800 can be exited.

In some embodiments, the method 800 can include at 810, determining fromwhich imaging sensor to display images in response to a signalindicative of a sensed light level in the welding environment.

In some embodiments, the method 800 can include at 812, displaying atleast one of information and images to assist the user of the wearablewelding apparatus in the performance of a weld in the weldingenvironment, wherein the information comprises at least weldingparameters of the welding environment including at least one of at leastone temperature in the welding environment, a feed speed of a weld wire,a position of a weld wire, a position of a weld torch, a current orvoltage of a power supply associated with the weld torch, and a dwelltime and the images comprise augmented reality images including at leastone of a thermal image overlaid on an image of the welding environmentand a path overlaid on an image of the welding environment along whichto perform a weld.

In some embodiments, the method 800 can include at 814 removing sparksfrom at least one image of the welding environment using at least theprocesses described above.

In some embodiments, the method 800 can include at 816 communicatingwith a welding station for determining at least operating parameters ofthe welding environment.

In some embodiments, the method 800 can include at 818 recording awelding process captured by the imaging sensors, where the recordedwelding process can be used for training.

In some embodiments, the method 800 can include at 820 evaluating imagesof a welding process captured by the imaging sensors and providingfeedback to a user regarding weld quality.

In some embodiments, the method 800 can include at 822 configuring theimaging sensors to capture images of the welding environment atdifferent frame rates to create images of the welding environment havingdifferent exposures.

FIG. 9 depicts a flow diagram of a method 900 for enhanced real-timewelding visualization in accordance with a stereo/three-dimensionalembodiment of the present principles. The method 900 begins at 902during which, in each of a left eye imaging assembly and a right eyeimaging assembly, incident light from a welding environment is splitusing a beam splitter into at least a first optical path and a secondoptical path having different light levels. The method can proceed to904.

At 904, in each of the at least first optical path and second opticalpath, images of the respective, split light are captured using arespective imaging sensor. The method 600 can proceed to 906.

At 906, images from the respective imaging sensors of the least firstoptical path and second optical path of the left eye imaging assemblyand the right eye imaging assembly are fused to create a left eye fusedimage and a right eye fused image. The method 900 can proceed to 908.

At 908, the left eye fused image is displayed at a location of a lefteye of a user of the wearable welding apparatus and the right eye fusedimage is displayed at a location of a right eye of the user of thewearable welding apparatus to provide a, high dynamic range, threedimensional representation of the welding environment and optimize depthperception. The method 900 can be exited.

Those skilled in the art will also appreciate that, while various itemsare illustrated as being stored in memory or on storage while beingused, these items or portions of them can be transferred between memoryand other storage devices for purposes of memory management and dataintegrity. Alternatively, in other embodiments some or all of thesoftware components can execute in memory on another device andcommunicate with the illustrated computer system via inter-computercommunication. Some or all of the system components or data structurescan also be stored (e.g., as instructions or structured data) on acomputer-accessible medium or a portable article to be read by anappropriate drive, various examples of which are described above. Insome embodiments, instructions stored on a computer-accessible mediumseparate from control unit 130 can be transmitted to control unit 130via transmission media or signals such as electrical, electromagnetic,or digital signals, conveyed via a communication medium such as anetwork and/or a wireless link. Various embodiments can further includereceiving, sending or storing instructions and/or data implemented inaccordance with the foregoing description upon a computer-accessiblemedium or via a communication medium. In general, a computer-accessiblemedium can include a storage medium or memory medium such as magnetic oroptical media, e.g., disk or DVD/CD-ROM, volatile or non-volatile mediasuch as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, and the like), ROM, and thelike.

The methods described herein may be implemented in software, hardware,or a combination thereof, in different embodiments. In addition, theorder of methods can be changed, and various elements can be added,reordered, combined, omitted or otherwise modified. All examplesdescribed herein are presented in a non-limiting manner. Variousmodifications and changes can be made as would be obvious to a personskilled in the art having benefit of this disclosure. Realizations inaccordance with embodiments have been described in the context ofparticular embodiments. These embodiments are meant to be illustrativeand not limiting. Many variations, modifications, additions, andimprovements are possible. Accordingly, plural instances can be providedfor components described herein as a single instance. Boundaries betweenvarious components, operations and data stores are somewhat arbitrary,and particular operations are illustrated in the context of specificillustrative configurations. Other allocations of functionality areenvisioned and can fall within the scope of claims that follow.Structures and functionality presented as discrete components in theexample configurations can be implemented as a combined structure orcomponent. These and other variations, modifications, additions, andimprovements can fall within the scope of embodiments as defined in theclaims that follow.

In the foregoing description, numerous specific details, examples, andscenarios are set forth in order to provide a more thoroughunderstanding of the present disclosure. It will be appreciated,however, that embodiments of the disclosure can be practiced withoutsuch specific details. Further, such examples and scenarios are providedfor illustration, and are not intended to limit the disclosure in anyway. Those of ordinary skill in the art, with the included descriptions,should be able to implement appropriate functionality without undueexperimentation.

References in the specification to “an embodiment,” etc., indicate thatthe embodiment described can include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Such phrases are notnecessarily referring to the same embodiment. Further, when a particularfeature, structure, or characteristic is described in connection with anembodiment, it is believed to be within the knowledge of one skilled inthe art to affect such feature, structure, or characteristic inconnection with other embodiments whether or not explicitly indicated.

Embodiments in accordance with the disclosure can be implemented inhardware, firmware, software, or any combination thereof. Embodimentscan also be implemented as instructions stored using one or moremachine-readable media, which may be read and executed by one or moreprocessors. A machine-readable medium can include any mechanism forstoring or transmitting information in a form readable by a machine(e.g., a control unit or a “virtual machine” running on one or morecontrol units). For example, a machine-readable medium can include anysuitable form of volatile or non-volatile memory.

Modules, data structures, and the like defined herein are defined assuch for ease of discussion and are not intended to imply that anyspecific implementation details are required. For example, any of thedescribed modules and/or data structures can be combined or divided intosub-modules, sub-processes or other units of computer code or data ascan be required by a particular design or implementation.

In the drawings, specific arrangements or orderings of schematicelements can be shown for ease of description. However, the specificordering or arrangement of such elements is not meant to imply that aparticular order or sequence of processing, or separation of processes,is required in all embodiments. In general, schematic elements used torepresent instruction blocks or modules can be implemented using anysuitable form of machine-readable instruction, and each such instructioncan be implemented using any suitable programming language, library,application-programming interface (API), and/or other softwaredevelopment tools or frameworks. Similarly, schematic elements used torepresent data or information can be implemented using any suitableelectronic arrangement or data structure. Further, some connections,relationships or associations between elements can be simplified or notshown in the drawings so as not to obscure the disclosure.

This disclosure is to be considered as exemplary and not restrictive incharacter, and all changes and modifications that come within theguidelines of the disclosure are desired to be protected.

Embodiments of the present principles include an apparatus for enhancedreal-time welding visualization in a welding environment including abeam splitter splitting incoming light from the welding environment intoat least a first optical path and a second optical path having differentlight levels, the at least first and second optical paths eachcomprising a respective imaging sensor, a control unit receiving andfusing the images from each of the respective imaging sensors in the atleast first optical path and second optical path to create a left eyefused image and a right eye fused image, and a display assemblydisplaying the left eye fused image at a location of a left eye of auser of the apparatus and the right eye fused image at a location of aright eye of the user of the apparatus.

The apparatus can further include a left eye imaging assembly and aright eye imaging assembly, each of the left eye imaging assembly andthe right eye imaging assembly including at least one beam splittersplitting incoming light from the welding environment into at least afirst optical path and a second optical path having different lightlevels, the at least first and second optical paths each comprising arespective imaging sensor, where images captured in the left eye imagingassembly are fused and displayed at the location of the left eye of theuser of the apparatus and images captured in the right eye imagingassembly are fused and displayed at the location of the right eye of theuser of the apparatus to provide a three dimensional representation ofthe welding environment.

In some embodiments, the apparatus can include at least one photo sensorfor sensing light levels in the welding environment and the control unitdetermines from which imaging sensor to display images based on a signalreceived from at least one of the at least one photo sensor and at leastone of the imaging sensors, the received signal indicative of a sensedlight level in the welding environment.

In addition or alternatively, in some embodiments, the apparatus canfurther include at least one temperature sensor for sensing temperaturesin the welding environment. In such embodiments and others, the controlunit causes a display, on the display assembly of a message indicativeof a sensed temperature in the welding environment in response to asignal communicated from the at least one temperature sensor indicativeof a sensed temperature in the welding environment.

In some embodiments, in the apparatus the display of the weldingenvironment includes a simultaneous display of images of an arc of aweld in progress, a welding wire, a tip of a welding torch, a puddle ofmolten metal just welded, a glowing bead of a weld that just solidified,the bead of the weld as the bead cools down and no longer glows, aregion of a sample to be welded, and a background region surrounding theweld area, wherein the simultaneous display includes all of the detailsof an individual display of images of the arc of a weld in progress, thewelding wire, the tip of the welding torch, the puddle of molten metaljust welded, the glowing bead of the weld that just solidified, the beadof the weld as the bead cools down and no longer glows, the region ofthe sample to be welded, and the background region surrounding the weldarea.

In addition or alternatively, in some embodiments, in the apparatus thecontrol unit causes a display, on the display assembly, of informationto assist in the performance of a weld. In such embodiments and others,the information to assist in the performance of a weld includes at leastone of an image generated by the control unit and welding parameters ofthe welding environment.

Alternatively or in addition, in the apparatus the control unit caninclude a memory for recording welding processes in the weldingenvironment. Even further, in some embodiments an apparatus inaccordance with the present principles can include at least one nearinfrared imaging sensor for enabling imaging of the welding environmentin the presence of smoke.

In at least some embodiments, the imaging sensors of the at least twooptical paths of the apparatus include printed circuit board cameras.Alternatively or in addition, the apparatus can include at least one ofa neutral density filter and a camera lens in at least one of the atleast first and second optical paths for further differentiating thelight level between the at least first and second optical paths.

Embodiments of the present principles include a method for enhancingreal-time welding visualization in a welding environment including, in awearable welding apparatus, splitting incident light from the weldingenvironment into at least a first optical path and a second optical pathhaving different light levels using at least one beam splitter, in eachof the at least first optical path and second optical path, capturingimages of the respective, split light using an imaging sensor, fusingthe images from the respective imaging sensors of the least firstoptical path and second optical path of the left eye imaging assemblyand the right eye imaging assembly to create a left eye fused image anda right eye fused image, and displaying the left eye fused image on adisplay at a location of a left eye of a user of the wearable weldingapparatus and the right eye fused image on a display at a location of aright eye of the user of the wearable welding apparatus.

Alternatively or in addition, in some embodiments, the incident lightfrom the welding environment is split into at least a first optical pathand a second optical path having different light levels using at leastone beam splitter in each of a left eye imaging assembly and a right eyeimaging assembly. The images captured in the left eye imaging assemblyare fused and displayed at the location of the left eye of the user ofthe apparatus and images captured in the right eye imaging assembly arefused and displayed at the location of the right eye of the user toprovide a three dimensional representation of the welding environment.

In some embodiments, the method can further include determining fromwhich imaging sensor to display images in response to a signalindicative of a sensed light level in the welding environment.

In some embodiments, the method can further include displaying at leastone of information and images to assist the user of the wearable weldingapparatus in the performance of a weld in the welding environment,wherein the information comprises at least welding parameters of thewelding environment including at least one of at least one temperaturein the welding environment, a feed speed of a weld wire, a position of aweld wire, a position of a weld torch, a current or voltage of a powersupply associated with the weld torch, and a dwell time and the imagescomprise augmented reality images including at least one of a thermalimage overlaid on an image of the welding environment and a pathoverlaid on an image of the welding environment along which to perform aweld.

In accordance with some embodiments of the present principle, the methodcan include removing sparks from at least one image of the weldingenvironment and alternatively or in addition, can also includecommunicating with a welding station for determining at least operatingparameters of the welding environment.

In some embodiments, for training or other purposes, the method caninclude recording a welding process captured by the imaging sensors. Insuch and other embodiments, the method can include evaluating images ofa welding process captured by the imaging sensors and providing feedbackto a user regarding weld quality.

In some embodiments in accordance with the present principles, themethod can include configuring the imaging sensors to capture images ofthe welding environment at different frame rates to create images of thewelding environment having different exposures.

Embodiments in accordance with the present principles include a methodfor enhanced real-time welding visualization and assistance in a weldingenvironment including splitting incoming light into a first optical pathhaving a minority of the split light and a second optical path having amajority of the split light using a beam splitter, and in the firstoptical path, focusing the minority of the split light from the beamsplitter using a high f-number lens, filtering the focused light fromthe high f-number lens using a neutral density filter, and capturingimages of the filtered light from the neutral density filter using afirst camera. In some embodiments the method can further include, in thesecond optical path, focusing the majority of the split light from thebeam splitter using a low f-number lens, capturing images of the lightfrom the low f-number lens using a second camera, fusing the images fromthe first camera and the second camera, and displaying at least aportion of the fused images as left eye display images and right eyedisplay images.

Alternatively, a method for enhanced real-time welding visualization andassistance in a welding environment includes in each of a left eyeimaging assembly and a right eye imaging assembly, splitting incominglight into a first optical path having a minority of the split light anda second optical path having a majority of the split light using a beamsplitter, and in the first optical path, focusing the minority of thesplit light from the beam splitter using a high f-number lens, filteringthe focused light from the high f-number lens using a neutral densityfilter, and capturing images of the filtered light from the neutraldensity filter using a first camera. In some embodiments the method canfurther include, in the second optical path, focusing the majority ofthe split light from the beam splitter using a low f-number lens,capturing images of the light from the low f-number lens using a secondcamera. The method can further include, fusing the images from the firstcamera and the second camera of the left eye imaging assembly and theright eye imaging assembly, and displaying at least a portion of thefused images as left eye display images and right eye display images.

1. An apparatus for enhanced real-time welding visualization in awelding environment, comprising: a beam splitter splitting incominglight from the welding environment into at least a first optical pathand a second optical path having different light levels, the at leastfirst and second optical paths each comprising a respective imagingsensor; a control unit receiving and fusing the images from each of therespective imaging sensors in the at least first optical path and secondoptical path to create a left eye fused image and a right eye fusedimage; and a display assembly displaying the left eye fused image at alocation of a left eye of a user of the apparatus and the right eyefused image at a location of a right eye of the user of the apparatus.2. The apparatus of claim 1, comprising: a left eye imaging assembly anda right eye imaging assembly, each of the left eye imaging assembly andthe right eye imaging assembly, comprising; at least one beam splittersplitting incoming light from the welding environment into at least afirst optical path and a second optical path having different lightlevels, the at least first and second optical paths each comprising arespective imaging sensor; wherein images captured in the left eyeimaging assembly are fused and then are displayed at the location of theleft eye of the user of the apparatus and images captured in the righteye imaging assembly are fused and displayed at the location of theright eye of the user of the apparatus to provide a three dimensionalrepresentation of the welding environment.
 3. The apparatus of claim 1,further comprising at least one photo sensor for sensing light levels inthe welding environment and the control unit determines from whichimaging sensor to display images based on a signal received from atleast one of the at least one photo sensor and at least one of theimaging sensors, the received signal indicative of a sensed light levelin the welding environment.
 4. The apparatus of claim 1, furthercomprising at least one temperature sensor for sensing temperatures inthe welding environment.
 5. The apparatus of claim 4, wherein thecontrol unit causes a display, on the display assembly of a messageindicative of a sensed temperature in the welding environment inresponse to a signal communicated from the at least one temperaturesensor indicative of a sensed temperature in the welding environment. 6.The apparatus of claim 1, wherein the displaying of the images of thewelding environment comprises a simultaneous display of images of an arcof a weld in progress, a welding wire a welding torch, a puddle ofmolten metal just welded, a glowing bead of a weld that just solidified,the bead of the weld as the bead cools down and no longer glows, aregion of a sample to be welded, and a background region surrounding theweld area, wherein the simultaneous display includes all of the detailsof an individual display of images of the arc of a weld in progress, thewelding wire, the tip of the welding torch, the puddle of molten metaljust welded, the glowing bead of the weld that just solidified, the beadof the weld as the bead cools down and no longer glows, the region ofthe sample to be welded, and the background region surrounding the weldarea.
 7. The apparatus of claim 1, wherein the control unit causes adisplay, on the display assembly, of information to assist in theperformance of a weld.
 8. The apparatus of claim 7, wherein theinformation to assist in the performance of a weld comprises at leastone of an image generated by the control unit and welding parameters ofthe welding environment.
 9. The apparatus of claim 1, wherein thecontrol unit comprises a memory for recording welding processes in thewelding environment.
 10. The apparatus of claim 1, further comprising atleast one near infrared imaging sensor for enabling imaging of thewelding environment in the presence of smoke.
 11. The apparatus of claim1, wherein the imaging sensors comprise printed circuit board cameras.12. The apparatus of claim 1, further comprising at least one of aneutral density filter and a camera lens in at least one of the at leastfirst and second optical paths for further differentiating the lightlevel between the at least first and second optical paths.
 13. A methodfor enhancing real-time welding visualization in a welding environment,the method comprising: in a wearable welding apparatus, splittingincident light from the welding environment into at least a firstoptical path and a second optical path having different light levelsusing at least one beam splitter; in each of the at least first opticalpath and second optical path, capturing images of the respective, splitlight using an imaging sensor; fusing the images from the respectiveimaging sensors of the at least first optical path and second opticalpath to create a left eye fused image and a right eye fused image; anddisplaying the left eye fused image on a display at a location of a lefteye of a user of the wearable welding apparatus and the right eye fusedimage on a display at a location of a right eye of the user of thewearable welding apparatus.
 14. The method of claim 13, comprising: ineach of a left eye imaging assembly and a right eye imaging assembly,splitting the incident light from the welding environment into at leasta first optical path and a second optical path having different lightlevels using at least one beam splitter; in each of the left eye imagingassembly and the right eye imaging assembly, capturing images in each ofthe at least first optical path and second optical path using an imagingsensor; fusing images from the respective imaging sensors of the atleast first optical path and second optical path of the left eye imagingassembly and the right eye imaging assembly to create a left eye fusedimage and a right eye fused image; and displaying the left eye fusedimage at a location of a left eye of a user of the wearable weldingapparatus and the right eye fused image is displayed at a location of aright eye of the user of the wearable welding apparatus.
 15. The methodof claim 13, further comprising determining from which imaging sensor todisplay images in response to a signal indicative of a sensed lightlevel in the welding environment.
 16. The method of claim 13, furthercomprising displaying at least one of information and images to assistthe user of the wearable welding apparatus in the performance of a weldin the welding environment, wherein the information comprises at leastwelding parameters of the welding environment including at least one ofat least one temperature in the welding environment, a feed speed of aweld wire, a position of a weld wire, a position of a weld torch, acurrent or voltage of a power supply associated with the weld torch, anda dwell time and the images comprise augmented reality images includingat least one of a thermal image overlaid on an image of the weldingenvironment and a path overlaid on an image of the welding environmentalong which to perform a weld.
 17. The method of claim 13, furthercomprising removing sparks from at least one image of the weldingenvironment.
 18. The method of claim 13, further comprisingcommunicating with a welding station for determining at least operatingparameters of the welding environment.
 19. (canceled)
 20. The method ofclaim 13, further comprising evaluating images of a welding processcaptured by the imaging sensors and providing feedback to a userregarding weld quality.
 21. The method of claim 13, further comprisingconfiguring the imaging sensors to capture images of the weldingenvironment at different frame rates to create images of the weldingenvironment having different exposures.